Method and apparatus for modifying the presentation of information based on the attentiveness level of a user

ABSTRACT

Methods, apparatuses, and computer program products are herein provided for modifying the presentation of information based on the attentiveness level of a user. A method may include determining a first attentiveness level of a user. The method may further include causing presentation of information on a display. The method may further include determining a second attentiveness level of the user following determination of the first attentiveness level of the user and while the information is being presented on the display. The method may further include causing, based on a relationship between the first attentiveness level and the second attentiveness level, modification of presentation of at least a portion of the information on the display. Corresponding apparatuses and computer program products are also provided.

TECHNOLOGICAL FIELD

Example embodiments of the present invention relate generally to userinterface technology and, more particularly, relate to methods,apparatuses, and computer program products for modifying thepresentation of information based on the attentiveness level of a user.

BACKGROUND

The modern communications era has brought about a tremendous expansionof wireline and wireless networks. Wireless and mobile networkingtechnologies have addressed related consumer demands, while providingmore flexibility and immediacy of information transfer. Concurrent withthe expansion of networking technologies, an expansion in computingpower has resulted in development of affordable computing devicescapable of taking advantage of services made possible by modernnetworking technologies. This expansion in computing power has led to areduction in the size of computing devices and given rise to a newgeneration of mobile devices that are capable of performingfunctionality that only a few years ago required processing power thatcould be provided only by the most advanced desktop computers.Consequently, mobile computing devices having a small form factor havebecome ubiquitous and are used to access network applications andservices by consumers of all socioeconomic backgrounds.

Increased functionality of mobile computing devices has led to increaseduse. Indeed, users often use these devices throughout the entire day,often while being fatigued. A fatigued user, however, may havedifficulty effectively operating or utilizing the device, such as notbeing able to easily see small text or focus on small objects. Thisproblem may be more significant with the use of pass-through displays(e.g., head-mounted displays, augmented reality glasses, video oroptical see-through displays, etc.), where the user may rely on thedevice for notification of important information regarding the user'ssurroundings (e.g., the user is driving a car).

BRIEF SUMMARY

As such, certain example embodiments of the present invention modify thepresentation of information on the display based on the attentivenesslevel of the user. In such a manner, the information being presented canbe optimized based on the attentiveness level of the user. For example,if a user is fatigued, an example embodiment of the present inventionmay make presentation of the information easier to perceive to aid afatigued user in recognizing the information. Thus, some embodiments ofthe present invention provide a dynamic system that modifies thepresented information based on the attentiveness level of the user inorder to optimize the user's experience.

Embodiments of the present invention provide methods, apparatuses, andcomputer program products for modifying the presentation of informationbased on the attentiveness level of a user. In one example embodiment, amethod includes determining a first attentiveness level of a user. Themethod further includes causing presentation of information on adisplay. The method further includes determining a second attentivenesslevel of the user following determination of the first attentivenesslevel of the user and while the information is being presented on thedisplay. The method further includes causing, based on a relationshipbetween the first attentiveness level and the second attentivenesslevel, modification of presentation of at least a portion of theinformation on the display.

In some embodiments, the method may further include causing modificationof presentation of the at least a portion of the information by causingmodification of presentation of the at least a portion of theinformation in an instance in which the second attentiveness level isdifferent than the first attentiveness level. Additionally oralternatively, the method may further include causing modification ofpresentation of the at least a portion of the information by causing theat least a portion of the information to be presented such that the atleast a portion of the information is easier to perceive. Additionallyor alternatively, the method may further include causing modification ofpresentation of the at least a portion of the information by causingmodification of at least one of font size or color scheme of the atleast a portion of the information. Additionally or alternatively, themethod may further include causing modification of presentation of theat least a portion of the information by causing at least a portion ofthe information presented on the display to be removed.

In some embodiments, the method may further include causing presentationof the information on the display by causing presentation of theinformation based on the first attentiveness level of the user.

In some embodiments, the method may further include determining thesecond attentiveness level by determining the second attentiveness levelbased at least in part on output provided by at least one of: a grazetracker, an accelerometer, or a heart rate monitor. Additionally oralternatively, the method may further include determining the secondattentiveness level by determining the second attentiveness level basedat least in part on detecting at least one of changes in movement orbehavior of the user's eyes or changes in movement or behavior of theuser's eyelids.

In another example embodiment, an apparatus includes at least oneprocessor and at least one memory storing computer program code, whereinthe at least one memory and stored computer program code are configured,with the at least one processor, to cause the apparatus to determine afirst attentiveness level of a user. The at least one memory and storedcomputer program code are configured, with the at least one processor,to further cause the apparatus to cause presentation of information on adisplay. The at least one memory and stored computer program code areconfigured, with the at least one processor, to further cause theapparatus to determine a second attentiveness level of the userfollowing determination of the first attentiveness level of the user andwhile the information is being presented on the display. The at leastone memory and stored computer program code are configured, with the atleast one processor, to further cause the apparatus to cause, based on arelationship between the first attentiveness level and the secondattentiveness level, modification of presentation of at least a portionof the information on the display.

In another example embodiment, a computer program product is provided.The computer program product of this example embodiment includes atleast one computer-readable storage medium having computer-readableprogram instructions stored therein. The program instructions of thisexample embodiment comprise program instructions configured to cause anapparatus to perform a method including determining a firstattentiveness level of a user. The method further includes causingpresentation of information on a display. The method further includesdetermining a second attentiveness level of the user followingdetermination of the first attentiveness level of the user and while theinformation is being presented on the display. The method furtherincludes causing, based on a relationship between the firstattentiveness level and the second attentiveness level, modification ofpresentation of at least a portion of the information on the display.

In another example embodiment, an apparatus is provided. The apparatuscomprises means for determining a first attentiveness level of a user.The apparatus further includes means for causing presentation ofinformation on a display. The apparatus further includes means fordetermining a second attentiveness level of the user followingdetermination of the first attentiveness level of the user and while theinformation is being presented on the display. The apparatus furtherincludes means for causing, based on a relationship between the firstattentiveness level and the second attentiveness level, modification ofpresentation of at least a portion of the information on the display.

BRIEF DESCRIPTION OF THE DRAWINGS

Having thus described some embodiments of the invention in generalterms, reference will now be made to the accompanying drawings, whichare not necessarily drawn to scale, and wherein:

FIG. 1 illustrates a block diagram of an apparatus with a user interfaceaccording to an example embodiment;

FIG. 2 is a schematic block diagram of a mobile terminal according to anexample embodiment;

FIG. 3A illustrates an example head-mounted display, wherein the displayis pass-through such that a city skyline can be seen through thedisplay, in accordance with an example embodiment of the presentinvention;

FIG. 3B illustrates an example windshield display for a car, wherein thedisplay is pass-through such that a city skyline can be seen through thedisplay, in accordance with an example embodiment of the presentinvention;

FIG. 4A illustrates an example device with a display, whereinapplication icons are presented on the display, in accordance with anexample embodiment described herein;

FIG. 4B illustrates an example car with a windshield with a pass-throughdisplay, wherein driving directions are presented on the pass-throughdisplay, in accordance with an example embodiment described herein;

FIG. 5A illustrates the device shown in FIG. 4A, wherein presentation ofthe application icons has been modified, in accordance with an exampleembodiment described herein;

FIG. 5B illustrates the device shown in FIG. 4B, wherein presentation ofthe driving directions has been modified, in accordance with an exampleembodiment described herein;

FIG. 6A illustrates the device shown in FIG. 4A, wherein some of theapplication icons have been removed from being presented on the display,in accordance with an example embodiment described herein;

FIG. 6B illustrates the device shown in FIG. 4B, wherein a portion ofthe sun shining through the pass-through display has been covered upfrom the perspective of the user, in accordance with an exampleembodiment described herein;

FIG. 7A illustrates the device shown in FIG. 4B, wherein an enlargedstop sign has been presented on the pass-through display over the stopsign from the perspective of the user, in accordance with an exampleembodiment described herein;

FIG. 7B illustrates the device shown in FIG. 4B, wherein enlarged textdirecting the user to “STOP” has been presented on the pass-throughdisplay over the stop sign from the perspective of the user, inaccordance with an example embodiment described herein;

FIG. 7C illustrates the device shown in FIG. 4B, wherein a portion ofthe sun shining through the pass-through display has been covered upfrom the perspective of the user and wherein the driving directions havebeen removed from being presented on the display, in accordance with anexample embodiment described herein;

FIG. 8 illustrates a flowchart according to an example method formodifying the presentation of information based on the attentivenesslevel of a user, in accordance with an example embodiment describedherein; and

FIG. 9 illustrates a flowchart according to another example method formodifying the presentation of information based on the attentivenesslevel of a user, in accordance with an example embodiment describedherein.

DETAILED DESCRIPTION

Some embodiments of the present invention will now be described morefully hereinafter with reference to the accompanying drawings, in whichsome, but not all embodiments of the invention are shown. Indeed, theinvention may be embodied in many different forms and should not beconstrued as limited to the embodiments set forth herein; rather, theseembodiments are provided so that this disclosure will satisfy applicablelegal requirements. Like reference numerals refer to like elementsthroughout.

As used herein, the terms “data,” “content,” “information” and similarterms may be used interchangeably to refer to singular or plural datacapable of being transmitted, received, displayed and/or stored inaccordance with various example embodiments. Thus, use of any such termsshould not be taken to limit the spirit and scope of the disclosure.

The term “computer-readable medium” as used herein refers to any mediumconfigured to participate in providing information to a processor,including instructions for execution. Such a medium may take many forms,including, but not limited to a non-transitory computer-readable storagemedium (e.g., non-volatile media, volatile media), and transmissionmedia. Transmission media include, for example, coaxial cables, copperwire, fiber optic cables, and carrier waves that travel through spacewithout wires or cables, such as acoustic waves and electromagneticwaves, including radio, optical and infrared waves. Signals includeman-made transient variations in amplitude, frequency, phase,polarization or other physical properties transmitted through thetransmission media. Examples of non-transitory computer-readable mediainclude a magnetic computer readable medium (e.g., a floppy disk, harddisk, magnetic tape, any other magnetic medium), an optical computerreadable medium (e.g., a compact disc read only memory (CD-ROM), adigital versatile disc (DVD), a Blu-Ray disc, or the like), a randomaccess memory (RAM), a programmable read only memory (PROM), an erasableprogrammable read only memory (EPROM), a FLASH-EPROM, or any othernon-transitory medium from which a computer can read. The termcomputer-readable storage medium is used herein to refer to anycomputer-readable medium except transmission media. However, it will beappreciated that where embodiments are described to use acomputer-readable storage medium, other types of computer-readablemediums may be substituted for or used in addition to thecomputer-readable storage medium in alternative embodiments.

Additionally, as used herein, the term ‘circuitry’ refers to (a)hardware-only circuit implementations (e.g., implementations in analogcircuitry and/or digital circuitry); (b) combinations of circuits andcomputer program product(s) comprising software and/or firmwareinstructions stored on one or more computer readable memories that worktogether to cause an apparatus to perform one or more functionsdescribed herein; and (c) circuits, such as, for example, amicroprocessor(s) or a portion of a microprocessor(s), that requiresoftware or firmware for operation even if the software or firmware isnot physically present. This definition of ‘circuitry’ applies to alluses of this term herein, including in any claims. As a further example,as used herein, the term ‘circuitry’ also includes an implementationcomprising one or more processors and/or portion(s) thereof andaccompanying software and/or firmware. As another example, the term‘circuitry’ as used herein also includes, for example, a basebandintegrated circuit or applications processor integrated circuit for amobile phone or a similar integrated circuit in a server, a cellularnetwork device, other network device, and/or other computing device.

FIG. 1 illustrates a block diagram of an apparatus 102 for modifying thepresentation of information based on the attentiveness level of a user.It will be appreciated that the apparatus 102 is provided as an exampleof one embodiment and should not be construed to narrow the scope orspirit of the invention in any way. In this regard, the scope of thedisclosure encompasses many potential embodiments in addition to thoseillustrated and described herein. As such, while FIG. 1 illustrates oneexample of a configuration of an apparatus for facilitating interactionwith a user interface, other configurations may also be used toimplement embodiments of the present invention.

The apparatus 102 may be embodied as either a fixed device or a mobiledevice such as a desktop computer, laptop computer, mobile terminal,mobile computer, mobile phone, mobile communication device, game device,digital camera/camcorder, audio/video player, television device, radioreceiver, digital video recorder, positioning device, a chipset, acomputing device comprising a chipset, any combination thereof, and/orthe like. In this regard, the apparatus 102 may comprise any computingdevice that comprises or is in operative communication with a display.In some example embodiments, the apparatus 102 is embodied as a mobilecomputing device, such as the mobile terminal illustrated in FIG. 2.

In this regard, FIG. 2 illustrates a block diagram of a mobile terminal10 representative of one example embodiment of an apparatus 102. Itshould be understood, however, that the mobile terminal 10 illustratedand hereinafter described is merely illustrative of one type ofapparatus 102 that may implement and/or benefit from various exampleembodiments of the invention and, therefore, should not be taken tolimit the scope of the disclosure. While several embodiments of thecomputing device are illustrated and will be hereinafter described forpurposes of example, other types of computing devices, such as mobiletelephones, mobile computers, personal digital assistants (PDAs),pagers, laptop computers, desktop computers, gaming devices, positioningdevices, tablet computers, televisions, e-papers, and other types ofelectronic systems, may employ various embodiments of the invention.

As shown, the mobile terminal 10 may include an antenna 12 (or multipleantennas 12) in communication with a transmitter 14 and a receiver 16.The mobile terminal 10 may also include a processor 20 configured toprovide signals to and receive signals from the transmitter andreceiver, respectively. The processor 20 may, for example, be embodiedas various means including circuitry, one or more microprocessors withaccompanying digital signal processor(s), one or more processor(s)without an accompanying digital signal processor, one or morecoprocessors, one or more multi-core processors, one or morecontrollers, processing circuitry, one or more computers, various otherprocessing elements including integrated circuits such as, for example,an ASIC (application specific integrated circuit) or FPGA (fieldprogrammable gate array), or some combination thereof. Accordingly,although illustrated in FIG. 2 as a single processor, in someembodiments the processor 20 comprises a plurality of processors. Thesesignals sent and received by the processor 20 may include signalinginformation in accordance with an air interface standard of anapplicable cellular system, and/or any number of different wireline orwireless networking techniques, comprising but not limited to Wi-Fi,wireless local access network (WLAN) techniques such as Institute ofElectrical and Electronics Engineers (IEEE) 802.11, 802.16, and/or thelike. In addition, these signals may include speech data, user generateddata, user requested data, and/or the like. In this regard, the mobileterminal may be capable of operating with one or more air interfacestandards, communication protocols, modulation types, access types,and/or the like. More particularly, the mobile terminal may be capableof operating in accordance with various first generation (1G), secondgeneration (2G), 2.5G, third-generation (3G) communication protocols,fourth-generation (4G) communication protocols, Internet ProtocolMultimedia Subsystem (IMS) communication protocols (e.g., sessioninitiation protocol (SIP)), and/or the like. For example, the mobileterminal may be capable of operating in accordance with 2G wirelesscommunication protocols IS-136 (Time Division Multiple Access (TDMA)),Global System for Mobile communications (GSM), IS-95 (Code DivisionMultiple Access (CDMA)), and/or the like. Also, for example, the mobileterminal may be capable of operating in accordance with 2.5G wirelesscommunication protocols General Packet Radio Service (GPRS), EnhancedData GSM Environment (EDGE), and/or the like. Further, for example, themobile terminal may be capable of operating in accordance with 3Gwireless communication protocols such as Universal MobileTelecommunications System (UMTS), Code Division Multiple Access 2000(CDMA2000), Wideband Code Division Multiple Access (WCDMA), TimeDivision-Synchronous Code Division Multiple Access (TD-SCDMA), and/orthe like. The mobile terminal may be additionally capable of operatingin accordance with 3.9G wireless communication protocols such as LongTerm Evolution (LTE) or Evolved Universal Terrestrial Radio AccessNetwork (E-UTRAN) and/or the like. Additionally, for example, the mobileterminal may be capable of operating in accordance withfourth-generation (4G) wireless communication protocols and/or the likeas well as similar wireless communication protocols that may bedeveloped in the future.

Some Narrow-band Advanced Mobile Phone System (NAMPS), as well as TotalAccess Communication System (TACS), mobile terminals may also benefitfrom embodiments of this invention, as should dual or higher mode phones(e.g., digital/analog or TDMA/CDMA/analog phones). Additionally, themobile terminal 10 may be capable of operating according to Wi-Fi orWorldwide Interoperability for Microwave Access (WiMAX) protocols.

It is understood that the processor 20 may comprise circuitry forimplementing audio/video and logic functions of the mobile terminal 10.For example, the processor 20 may comprise a digital signal processordevice, a microprocessor device, an analog-to-digital converter, adigital-to-analog converter, and/or the like. Control and signalprocessing functions of the mobile terminal may be allocated betweenthese devices according to their respective capabilities. The processormay additionally comprise an internal voice coder (VC) 20 a, an internaldata modem (DM) 20 b, and/or the like. Further, the processor maycomprise functionality to operate one or more software programs (e.g.,applications), which may be stored in memory. For example, the processor20 may be capable of operating a connectivity program, such as a webbrowser. The connectivity program may allow the mobile terminal 10 totransmit and receive web content, such as location-based content,according to a protocol, such as Wireless Application Protocol (WAP),hypertext transfer protocol (HTTP), and/or the like. The mobile terminal10 may be capable of using a Transmission Control Protocol/InternetProtocol (TCP/IP) to transmit and receive web content across theinternet or other networks.

The mobile terminal 10 may also comprise a user interface including, forexample, an earphone or speaker 24, a ringer 22, a microphone 26, adisplay 28, a user input interface, and/or the like, which may beoperationally coupled to the processor 20. In this regard, the processor20 may comprise user interface circuitry configured to control at leastsome functions of one or more elements of the user interface, such as,for example, the speaker 24, the ringer 22, the microphone 26, thedisplay 28, and/or the like. The processor 20 and/or user interfacecircuitry comprising the processor 20 may be configured to control oneor more functions of one or more elements of the user interface throughcomputer program instructions (e.g., software and/or firmware) stored ona memory accessible to the processor 20 (e.g., volatile memory 40,non-volatile memory 42, and/or the like). Although not shown, the mobileterminal may comprise a battery for powering various circuits related tothe mobile terminal, for example, a circuit to provide mechanicalvibration as a detectable output. The display 28 of the mobile terminalmay be of any type appropriate for the electronic device in questionwith some examples including a plasma display panel (PDP), a liquidcrystal display (LCD), a light-emitting diode (LED), an organiclight-emitting diode display (OLED), a pass-through display, aprojector, a holographic display or the like. The display 28 may, forexample, comprise a three-dimensional touch display, examples of whichwill be described further herein below. The user input interface maycomprise devices allowing the mobile terminal to receive data, such as akeypad 30, a touch display (e.g., some example embodiments wherein thedisplay 28 is configured as a touch display), a joystick (not shown),and/or other input device. In embodiments including a keypad, the keypadmay comprise numeric (0-9) and related keys (#, *), and/or other keysfor operating the mobile terminal 10. Alternatively or additionally, thekeypad 30 may include a conventional QWERTY keypad arrangement.

The mobile terminal 10 may comprise memory, such as a subscriberidentity module (SIM) 38, a removable user identity module (R-UIM),and/or the like, which may store information elements related to amobile subscriber. In addition to the SIM, the mobile terminal maycomprise other removable and/or fixed memory. The mobile terminal 10 mayinclude volatile memory 40 and/or non-volatile memory 42. For example,volatile memory 40 may include Random Access Memory (RAM) includingdynamic and/or static RAM, on-chip or off-chip cache memory, and/or thelike. Non-volatile memory 42, which may be embedded and/or removable,may include, for example, read-only memory, flash memory, magneticstorage devices (e.g., hard disks, floppy disk drives, magnetic tape,etc.), optical disc drives and/or media, non-volatile random accessmemory (NVRAM), and/or the like. Like volatile memory 40 non-volatilememory 42 may include a cache area for temporary storage of data. Thememories may be non-transitory and may store one or more softwareprograms, instructions, pieces of information, data, and/or the likewhich may be used by the mobile terminal for performing functions of themobile terminal. For example, the memories may comprise an identifier,such as an international mobile equipment identification (IMEI) code,capable of uniquely identifying the mobile terminal 10.

Returning to FIG. 1, in an example embodiment, the apparatus 102includes various means for performing the various functions hereindescribed. These means may comprise one or more of a processor 110,memory 112, communication interface 114, user interface 116, sensor 118,or user interface (UI) control circuitry 122. The means of the apparatus102 as described herein may be embodied as, for example, circuitry,hardware elements (e.g., a suitably programmed processor, combinationallogic circuit, and/or the like), a computer program product comprisingcomputer-readable program instructions (e.g., software or firmware)stored on a computer-readable medium (e.g. memory 112) that isexecutable by a suitably configured processing device (e.g., theprocessor 110), or some combination thereof.

In some example embodiments, one or more of the means illustrated inFIG. 1 may be embodied as a chip or chip set. In other words, theapparatus 102 may comprise one or more physical packages (e.g., chips)including materials, components and/or wires on a structural assembly(e.g., a baseboard). The structural assembly may provide physicalstrength, conservation of size, and/or limitation of electricalinteraction for component circuitry included thereon. In this regard,the processor 110, memory 112, communication interface 114, and/or UIcontrol circuitry 122 may be embodied as a chip or chip set. Theapparatus 102 may therefore, in some cases, be configured to or maycomprise component(s) configured to implement embodiments of the presentinvention on a single chip or as a single “system on a chip.” As such,in some cases, a chip or chipset may constitute means for performing oneor more operations for providing the functionalities described hereinand/or for enabling user interface navigation with respect to thefunctionalities and/or services described herein.

The processor 110 may, for example, be embodied as various meansincluding one or more microprocessors with accompanying digital signalprocessor(s), one or more processor(s) without an accompanying digitalsignal processor, one or more coprocessors, one or more multi-coreprocessors, one or more controllers, processing circuitry, one or morecomputers, various other processing elements including integratedcircuits such as, for example, an ASIC or FPGA, one or more other typesof hardware processors, or some combination thereof. Accordingly,although illustrated in FIG. 1 as a single processor, in someembodiments the processor 110 comprises a plurality of processors. Theplurality of processors may be in operative communication with eachother and may be collectively configured to perform one or morefunctionalities of the apparatus 102 as described herein. The pluralityof processors may be embodied on a single computing device ordistributed across a plurality of computing devices collectivelyconfigured to function as the apparatus 102. In embodiments wherein theapparatus 102 is embodied as a mobile terminal 10, the processor 110 maybe embodied as or comprise the processor 20 (shown in FIG. 2). In someexample embodiments, the processor 110 is configured to executeinstructions stored in the memory 112 or otherwise accessible to theprocessor 110. These instructions, when executed by the processor 110,may cause the apparatus 102 to perform one or more of thefunctionalities of the apparatus 102 as described herein. As such,whether configured by hardware or software methods, or by a combinationthereof, the processor 110 may comprise an entity capable of performingoperations according to embodiments of the present invention whileconfigured accordingly. Thus, for example, when the processor 110 isembodied as an ASIC, FPGA or the like, the processor 110 may comprisespecifically configured hardware for conducting one or more operationsdescribed herein. Alternatively, as another example, when the processor110 is embodied as an executor of instructions, such as may be stored inthe memory 112, the instructions may specifically configure theprocessor 110 to perform one or more algorithms and operations describedherein.

The memory 112 may comprise, for example, volatile memory, non-volatilememory, or some combination thereof. In this regard, the memory 112 maycomprise a non-transitory computer-readable storage medium. Althoughillustrated in FIG. 1 as a single memory, the memory 112 may comprise aplurality of memories. The plurality of memories may be embodied on asingle computing device or may be distributed across a plurality ofcomputing devices collectively configured to function as the apparatus102. In various example embodiments, the memory 112 may comprise a harddisk, random access memory, cache memory, flash memory, a compact discread only memory (CD-ROM), digital versatile disc read only memory(DVD-ROM), an optical disc, circuitry configured to store information,or some combination thereof. In embodiments wherein the apparatus 102 isembodied as a mobile terminal 10, the memory 112 may comprise thevolatile memory 40 and/or the non-volatile memory 42 (shown in FIG. 2).The memory 112 may be configured to store information, data,applications, instructions, or the like for enabling the apparatus 102to carry out various functions in accordance with various exampleembodiments. For example, in some example embodiments, the memory 112 isconfigured to buffer input data for processing by the processor 110.Additionally or alternatively, the memory 112 may be configured to storeprogram instructions for execution by the processor 110. The memory 112may store information in the form of static and/or dynamic information.The stored information may include, for example, images, content, mediacontent, user data, application data, and/or the like. This storedinformation may be stored and/or used by the UI control circuitry 122during the course of performing its functionalities.

The communication interface 114 may be embodied as any device or meansembodied in circuitry, hardware, a computer program product comprisingcomputer readable program instructions stored on a computer readablemedium (e.g., the memory 112) and executed by a processing device (e.g.,the processor 110), or a combination thereof that is configured toreceive and/or transmit data from/to another computing device. In someexample embodiments, the communication interface 114 is at leastpartially embodied as or otherwise controlled by the processor 110. Inthis regard, the communication interface 114 may be in communicationwith the processor 110, such as via a bus. The communication interface114 may include, for example, an antenna, a transmitter, a receiver, atransceiver and/or supporting hardware or software for enablingcommunications with one or more remote computing devices. In embodimentswherein the apparatus 102 is embodied as a mobile terminal 10, thecommunication interface 114 may be embodied as or comprise thetransmitter 14 and receiver 16 (shown in FIG. 2). The communicationinterface 114 may be configured to receive and/or transmit data usingany protocol that may be used for communications between computingdevices. In this regard, the communication interface 114 may beconfigured to receive and/or transmit data using any protocol that maybe used for transmission of data over a wireless network, wirelinenetwork, some combination thereof, or the like by which the apparatus102 and one or more computing devices may be in communication. As anexample, the communication interface 114 may be configured to receiveand/or otherwise access content (e.g., web page content, streaming mediacontent, and/or the like) over a network from a server or other contentsource. The communication interface 114 may additionally be incommunication with the memory 112, user interface 116, sensor 118,and/or UI control circuitry 122, such as via a bus.

The sensor 118 may be in communication with the processor 110, userinterface 116, and/or UI control circuitry 122. In embodiments whereinthe apparatus 102 is embodied as a mobile terminal 10, the sensor 118may be embodied as or comprise the sensor 18 (shown in FIG. 2). In someembodiments, the sensor 118 may be configured detect changes in movementor behavior of a user's eyes. In some embodiments, the sensor 118 may beconfigured to track a user's gaze, such as by detecting the locationand/or focus point of the user's eyes. For example, the sensor 118 maybe configured to transmit a beam or other signal (e.g., an infraredlight) that bounces off the user's eyes (e.g., the user's cornea). Thesensor 118 may also be configured to receive the beam or other signalupon its return. Additionally or alternatively, the sensor 118 may useother functionality including, but not limited to pictures and/or videotaken with a camera and/or detecting change in the user's eye-musclerelated electrical activity, among others, to track the user's gaze.Along these same lines, the sensor 118 may be configured to detect thesize of a user's pupil in order to determine the user's focus point in athree-dimensional environment (e.g., depth of the user's focus point).In some embodiments, the sensor 118 may provide output (e.g., signals,beams, pictures, videos, etc.) to the processor 110 to enable theprocessor 110 to determine certain circumstances regarding the user'sgaze (e.g., location, focus point, etc.). This information may be usedto determine changes in movement or behavior of the user's gaze.Similarly, the sensor 118 may be configured to monitor other features ofthe user, such as the movement of the user's eyelids, includingexcessive blinking or heavy-lidded behavior, among others. In someembodiments, the sensor 118 may comprise a gaze tracker. In such anembodiment, the sensor 118 may be configured to provide outputindicative of the any of the above detections to the apparatus 102(e.g., the processor 110), such as for determination of theattentiveness level of the user.

Additionally, in some embodiments, the sensor 118 may comprise a heartrate monitor to detect the heart rate of the user. In such anembodiment, the sensor 118 may be configured to provide an output to theapparatus 102 (e.g., the processor 110) to determine the heart rate ofthe user, such as for determination of the attentiveness level of theuser. Similarly, in some embodiments, the sensor 118 may comprise anaccelerometer to detect movement that may be indicative of the fatiguelevel of the user. For example, a head-mounted display may include anaccelerometer that may be configured to detect if a user drops theirhead, such as may be indicative that the user is falling asleep, such asfor determination of the attentiveness level of the user.

Additionally, in some embodiments, the sensor 118 may comprise afront-facing camera. In some embodiments, a front-facing camera may beuseful for an apparatus with a pass-through display to help detect(e.g., with pictures, videos, etc.) the environment the user is lookingat through the display (e.g., from the perspective of the user). Indeed,in some embodiments, the front-facing camera may provide output to theapparatus 102 (e.g., the processor 110), which may determine theenvironment, such as for replication and/or modification on thepass-through display.

Additionally, in some embodiments, the sensor 118 may comprise othertypes of sensors. For example, a galvanic skin response sensor may beused to measure the electrical conductance of the skin of the user. Sucha sensor may be positioned near the user's eyes to detect changes in theuser's gaze. Additionally or alternatively, the sensor 118 may comprisea sensor for detecting the body temperature of the user. Changes in bodytemperature may help determine the attentiveness level of the user.

As used herein, example embodiments of the apparatus 102 may include anytype of sensor 118 or combination of sensors 118, such as a gazetracker, proximity sensor, light sensor, gyroscope, camera, heart ratemonitor, and/or accelerometer, among others.

The user interface 116 may be in communication with the processor 110 toreceive an indication of a user input and/or to provide an audible,visual, mechanical, or other output to a user. As such, the userinterface 116 may include, for example, a keyboard, a mouse, a joystick,a display, a touch screen display, a microphone, a speaker, and/or otherinput/output mechanisms. In some embodiments, a display may refer todisplay on a screen, on a wall, on glasses (e.g., near-eye-display), inthe air, etc. In embodiments wherein the apparatus 102 is embodied as amobile terminal 10, the user interface 116 may be embodied as orcomprise the display 28 and keypad 30 (shown in FIG. 2). The userinterface 116 may be in communication with the memory 112, communicationinterface 114, sensor 118, and/or UI control circuitry 122, such as viaa bus.

In some embodiments, the user interface 116 may comprise a pass-throughdisplay. Likewise, in embodiments wherein the apparatus 102 is embodiedas a mobile terminal 10, the display 28 may be embodied as apass-through display. In some embodiments, a pass-through display may beconfigured to present information (e.g., text, icons, pictures, videos,visual attractants, etc.) in an at least partially transparent form(e.g., image overlay) such that a user may be able to see through theinformation being presented.

As such, some example embodiments may provide for an apparatus (e.g.,apparatus 102) for providing input through a device comprising adisplay, such as the glasses 150 shown in FIG. 3A or the windshield 170shown in FIG. 3B. The apparatus with a pass-through display may providea visual overlay of information on a substantially transparent displaysurface, such as through lenses that appear to be normal optical glasslenses. This visual overlay allows a user to view objects and people intheir typical, un-obscured field of view while providing additionalinformation that may be displayed on the pass-through display. Thevisual overlay of the information may be of various opacity ranging fromtransparent (or 0% opacity) to opaque (or 100% opacity). For example,the information presented on the pass-through display may, in somecases, fully occlude the user's vision beyond the information presentedon the display by being presented in a substantially opaque manner on asubstantially transparent display. In some embodiments, the presentedinformation may be partially transparent such that the wearer is able tosee the environment beyond the pass-through display. The degree oftransparency may be variable from fully transparent, where theinformation is not shown, to fully opaque or non-transparent, or anydegree therebetween.

Example embodiments may also present information at the edges of thepass-through display. For example, the pass-through display of oneembodiment may include a central area of the display which issubstantially transparent with less transparent and possibly opaqueinformation being presented around the substantially transparent area.Such an embodiment may allow a user to view their environment (e.g.,environment information) through the central area, while also providinginformation for the user to view peripherally thereabout.

Presentation of information on a pass-through display coupled withobject recognition allows for dynamically interactive information to bepresented to a user. Apparatuses with pass-through displays allow forthe presentation of two-dimensional (2-D) and/or three-dimensional (3-D)visual elements. Two-dimensional graphical elements rendered in theplane of the display can present the user with a user interface, statuselements, or other information as would be viewed on a typical computerscreen or display of a mobile terminal (e.g., display 28 of mobileterminal 10). Similarly, three-dimensional graphical elements may berendered on top of the display or on top of the environment and seenthrough the pass-through display. In such a manner, these visualelements can communicate messages, alert the user with a notification,render an application that is currently in use, etc. Additionally, thesevisual elements may provide images regarding an object that a user ofthe pass-through display is viewing through the display, such as byidentifying a point-of-interest or landmark that a user is viewing. Suchidentification may be accomplished by various means including objectrecognition software or object recognition in conjunction with locationidentification (e.g., via Global Positioning System (GPS) signals) ofthe apparatus (e.g., apparatus 102) or the device that embodies theapparatus, such as mobile terminal 10.

Additionally, in some embodiments, the apparatus (e.g., apparatus 102),may be multi-modal so as to provide additional feedback to the user. Forexample, the apparatus may provide audio feedback in addition to thepresentation of information. Additionally or alternatively, theapparatus may provide tactile feedback (e.g., vibration) to the user.

Some examples of apparatuses (e.g., apparatus 102) with a pass-throughdisplay are illustrated in FIGS. 3A and 3B. FIG. 3A illustrates anexample head-mounted display (e.g., glasses 150) that includes a leftlens 154 and a right lens 152. Such left and right lens 152, 154 may beconfigured to interact with a user's left eye and right eyerespectively. Additionally, the glasses 150 may include a frame 151 thatis configured to engage with a user's head (e.g., ears) to mount to theuser's head.

In some embodiments, the glasses 150 may comprise (or be incommunication with) a processor (e.g., processor 110) that is configuredto control presentation of information. For example, in the depictedembodiment of the glasses 150, the processor may be configured tocontrol presentation of information to create a left lens display 164and a right lens display 162. In some embodiments, the informationpresented on the left lens display 164 and right lens display 162 may beindependent and/or they may be complementary.

As illustrated in FIG. 3A, a user (not shown) may look through the leftlens 154 and right lens 152 and the left lens display 164 and right lensdisplay 162 presented thereupon to see information, such as a cityskyline 190. Additionally, in some embodiments, the glasses 150 may beconfigured to cause presentation on the left lens display 164 and/orright lens display 162. In the depicted embodiment, the description“CITY SKYLINE” 180 is being presented to the user in the left lensdisplays 164 and right lens display 162 to indicate that the user islooking at the skyline of the city (e.g., through the left lens 154 andright lens 152). Though words are shown as being presented on thedisplay, some embodiments of the present invention contemplatepresenting other types of information on the display (e.g., pictures,videos, notifications, visual attractants, etc.). Additionally, thoughthe words shown in FIG. 3A are not transparent, some embodiments of thepresent invention contemplate presentation of at least partiallytransparent information on the display.

FIG. 3B illustrates an example car 170 that may embody the apparatus 102with a windshield with a pass-through display. In the depictedembodiment, the car 170 includes a dashboard 172, steering wheel 171,rear-view mirror 173 and windshield 175. The windshield 175 includes adisplay 176. In some embodiments, the car 170 may comprise (or be incommunication with) a processor (e.g., processor 110) that is configuredto control presentation of information. For example, in the depictedembodiment of the car 170, the processor may be configured to controlpresentation of information on the display 176 of the windshield 175.

As illustrated in FIG. 3B, a user (not shown) may look through thewindshield 175 and the display 176 to see environment information, suchas a city skyline 190. Additionally, in some embodiments, the car 170may be configured to cause presentation of information on the display176. In the depicted embodiment, the description “CITY SKYLINE” 180 isbeing presented to the user on the display 176 to indicate that the useris looking at the skyline of the city (e.g., through the windshield175). Though words are shown as being presented on the display, someembodiments of the present invention contemplate presenting other typesof information on the display (e.g., pictures, videos, notifications,visual attractants, etc.). Additionally, though the words shown in FIG.3A are not transparent, some embodiments of the present inventioncontemplate presentation of at least partially transparent informationon the display.

It should also be noted that while FIGS. 3A and 3B each illustrate oneexample of an apparatus with a pass-through display, numerous otherapparatuses configured with a pass-through display are contemplated forembodiments of the present invention (e.g., a helmet visor, a cockpitwindshield, etc.). Moreover, embodiments of the present invention arenot meant to be limited to apparatuses with pass-through displays, andmay be useful with other example apparatuses, such as any apparatus 102(e.g., mobile terminal 10) described herein (e.g., a mobile computingdevice, a fixed computing device, etc.) having or associated with thedisplay.

Returning to FIG. 1, the UI control circuitry 122 may be embodied asvarious means, such as circuitry, hardware, a computer program productcomprising computer readable program instructions stored on a computerreadable medium (e.g., the memory 112) and executed by a processingdevice (e.g., the processor 110), or some combination thereof and, insome embodiments, is embodied as or otherwise controlled by theprocessor 110. In some example embodiments wherein the UI controlcircuitry 122 is embodied separately from the processor 110, the UIcontrol circuitry 122 may be in communication with the processor 110.The UI control circuitry 122 may further be in communication with one ormore of the memory 112, communication interface 114, sensor 118, and/oruser interface 116, such as via a bus.

The UI control circuitry 122 may be configured to receive user inputfrom a user interface 116, such as a touch display. The user input orsignal may carry positional information indicative of the user input. Inthis regard, the position may comprise a position of the user input in atwo-dimensional space, which may be relative to the surface of the touchdisplay user interface. For example, the position may comprise acoordinate position relative to a two-dimensional coordinate system(e.g., an X and Y axis), such that the position may be determined.Accordingly, the UI control circuitry 122 may determine a position ofthe user input such as for determining a portion of the display to whichthe user input correlates.

In some embodiments, the apparatus 102 may be configured to receiveother forms of user input (e.g., speech, gestures, etc.). Similarly,receipt of touch input by a user should not be limited to the displayand may include any portion of the apparatus 102.

In some embodiments, a touch display may be configured to enable thedetection of a hovering gesture input. A hovering gesture input maycomprise a gesture input to the touch display without making physicalcontact with a surface of the touch display, such as a gesture made in aspace some distance above/in front of the surface of the touch display.As an example, the touch display may comprise a projected capacitivetouch display, which may be configured to enable detection ofcapacitance of a finger or other input object by which a gesture may bemade without physically contacting a display surface. As anotherexample, the touch display may be configured to enable detection of ahovering gesture input through use of acoustic wave touch sensortechnology, electromagnetic touch sensing technology, near field imagingtechnology, optical sensing technology, infrared proximity sensingtechnology, some combination thereof, or the like.

A user's attentiveness level may vary and is often based on a number offactors, including the current level of fatigue of a user. For example,a user may be wide awake in the morning with a high energy level, suchas after a cup of coffee. Alternatively, a user may be less attentivewhen the user is tired, such after a long day of work.

The apparatus 102 may be configured to determine an attentiveness levelof a user. For example, the apparatus 102 may be configured to determinea first attentiveness level of a user. In some embodiments, theapparatus 102 may be configured to determine the attentiveness level ofa user with sensor 118, which, as noted above, may comprise a gazetracker. In such embodiments, the sensor 118 may be configured toprovide an output, such as to the processor 110, and the apparatus 102,such as the processor 110, may determine the attentiveness level of theuser, based at least in part on the output of the sensor 118.

In some embodiments, the apparatus 102, such as the processor 110, maybe configured to determine an attentiveness level of a user based atleast in part on detecting at least one of changes in movement orbehavior of the user's eyes or changes in movement or behavior of theuser's eyelids. As noted above, the apparatus 102 may comprise a sensor118 configured to monitor the user's eyes and/or eyelids, includingchanges in movement or behavior of the user's eyes and/or eyelids. Insuch an embodiment, the apparatus 102, such as the processor 110, maymonitor the user's eyes and/or eyelids for changes that indicate achange in the attentiveness level of the user. For example, a user'seyes may be quick to move between data (e.g., presented informationand/or environment information) such that the apparatus 102, such as theprocessor 110, may determine a high attentiveness level. However, if theuser becomes fatigued then the user's eyes may not be as quick to movebetween data, thereby indicating that the user may be fatigued and,thus, the apparatus 102, such as the processor 110, may determine a lowattentiveness level. Along these same lines, a user's eyelids may blinkquickly when a user is alert, such that the apparatus 102, such as theprocessor 110, may determine a high attentiveness level. However, once auser becomes fatigued, the user's eyelids may take longer to blink(e.g., exhibit heavy-lidded behavior), thereby indicating that the usermay be fatigued and, thus, the apparatus 102, such as the processor 110,may determine a low attentiveness level.

In some embodiments, the apparatus 102, such as the processor 110, maybe configured to determine an attentiveness level of a user based atleast in part on output provided by at least one of: a gaze tracker, anaccelerometer, or a heart rate monitor. For example, as noted above, agaze tracker (such as an example embodiment of sensor 118) may detectchanges in movement or behavior of a user's eyes and/or eyelids, therebyindicating an attentiveness level of a user. Additionally oralternatively, an accelerometer (such as an example embodiment of sensor118) may detect movement of the user and/or movement of the apparatus102 that indicates the attentiveness level of the user. For example, auser's head may bob up and down if they are falling asleep. Theapparatus 102 may be configured to detect this movement with anaccelerometer, such as when the apparatus 102 comprises a head-mounteddisplay (shown in FIG. 3A) or a mobile device (e.g., mobile terminal10). A heart rate monitor (such as an example embodiment of sensor 118)may also be used to determine a user's attentiveness level. For example,the heart rate monitor may detect the heart rate of the user, such thata quickened heart rate may indicate an alert user, while a slowed heartrate may indicate a fatigued user.

While some embodiments of the present invention may determine anattentiveness level based on a discrete event (e.g., detecting a headbob, the user not opening their eyelids for a certain amount of time,etc.), other embodiments may determine an attentiveness level based oncontinuous or repeated monitoring of overall changes or trends (e.g.,the reaction time of the user to recognize presented information orenvironment information decreases over time).

As used herein, the attentiveness level may be based on indications ofthe current alert and/or fatigue level of a user. Additionally, theattentiveness level may be defined by any absolute or relative qualifier(e.g., numbers, high/low, etc.). For example, a user that is highlyalert may be determined to have an attentiveness level that is high.This same user, however, such as after a long day of working, may becomefatigued and, thus, the apparatus 102, such as the processor 110, maydetermine that the user has an attentiveness level that is low.Similarly, the attentiveness level may be based on a number system(e.g., 0-10), such that a highly alert user may be determined to have anattentiveness level of 9 and a fatigued user may be determined to havean attentiveness level of 3.

In some embodiments, the apparatus 102, such as the processor 110, maybe configured to cause presentation of information on a display. Theinformation presented on the display may be based on the operation beingperformed by the apparatus 102. For example, with reference to FIG. 4A,a computing device 200 (such as an example embodiment of apparatus 102)may include a display 208. Icons, such as the Mail icon 210 or Phoneicon 225 may be presented as example information on the display 208.Additionally, in the depicted embodiment, a title of each icon ispresented (e.g., “Mail” 212 for the Mail icon 210 and “Phone” 227 forthe Phone icon 225).

In another example embodiment, with reference to FIG. 4B, an apparatus(e.g., apparatus 102) associated with a car 250 and a windshield 252,may include a pass-through display 258. The car 250 may also include asteering wheel 251 and a rear-view mirror 273. Environment information,such as the sun 293 and a stop sign 290 may be visible through thepass-through display 258. Additionally, the apparatus associated withthe car 250 may cause presentation of information on the pass-throughdisplay 258. In the depicted embodiment, directions for a user (e.g.,driver of the car 250) to “STOP THEN TURN RIGHT” 275 with a right turnarrow 277 are presented on the pass-through display 258.

In some embodiments, the apparatus 102, such as the processor 110, maybe configured to cause presentation of information on the display basedon the first attentiveness level of the user. For example, in someembodiments, the apparatus 102, such as the processor 110, may take intoaccount the determined first attentiveness level of the user whencausing presentation of the information on the display. In such aregard, as will be described in greater detail herein, the informationcan be presented in the most effective manner to the user. As anexample, with reference to FIG. 4A, an alert user may not require largeicons or large text for use of the device 200. It may be desirable,instead, to minimize the space on the display 208 devoted to such iconsor text in favor of other icons or applications. Similarly, withreference to FIG. 4B, an alert user may not require large text or alarge right turn arrow for recognition purposes. Moreover, it may bedesirable to minimize this information in favor of environmentinformation that may be otherwise blocked (at least partially) by thepresented information.

As noted above, a user's attentiveness level may change during use ofcomputing devices, such as apparatus 102 (e.g., extended use of suchdevices may lead to fatigue in a user). Additionally, a user may becomefatigued for many other reasons. Additionally, in some cases, a user maybe distracted, which may affect the attentiveness level of the user. Theattentiveness level of a user may affect how well a user is able toutilize a device. For example, a fatigued user may have difficultyperforming operations on the device, such as focusing on small text orkeeping focused on small objects. As such, some embodiments of thepresent invention seek to monitor the attentiveness level of the userand modify presentation of the information based on the attentivenesslevel of the user. In such a manner, some embodiments may provide adynamic display that adapts to the current attentiveness level of theuser.

In some embodiments, the apparatus 102, such as the processor 110, maybe configured to determine a second attentiveness level of the user. Insome embodiments, the apparatus 102, such as the processor 110, may beconfigured to determine a second attentiveness level of the userfollowing determination of the first attentiveness level of the user andwhile the information is being presented on the display. For example, asnoted above, the apparatus 102, such as the processor 110, may beconfigured to determine an attentiveness level of a user based on thedetection of certain characteristics regarding the user (e.g., changesin movement or behavior of the user's eyes and/or eyelids, heart rate ofthe user, movement of the user or apparatus, among others). Thismonitoring may occur over time after determination of the firstattentiveness level, such as during presentation of information on adisplay of the apparatus 102. Additionally, in some embodiments, otherfactors may be used for determination of the attentiveness level of theuser. For example, the attentiveness level of the user may be determinedbased on aggregate data from multiple users or canonical data fromanalysis of that aggregate data. Thus, social norms may be relevant tothe determination of the attentiveness level of the user.

In some embodiments, the apparatus 102, such as the processor 110, maybe configured to cause, based on a relationship between the firstattentiveness level and the second attentiveness level, modification ofpresentation of at least a portion of the information on the display. Insuch a regard, the apparatus 102, such as the processor 110, may beconfigured to adapt the presentation of the information based on achange in the attentiveness level of the user.

For example, in some embodiments, the apparatus 102, such as theprocessor 110, may be configured to compare the first attentivenesslevel and the second attentiveness level. In such an embodiment, in thecase that the user may have become fatigued since the determination ofthe first attentiveness level, the apparatus 102, such as the processor110, may determine that the second attentiveness level is lower than thefirst attentiveness level. In such a circumstance, the apparatus 102,such as the processor 110, may be configured to cause a modification ofpresentation of at least a portion of the information such that at leasta portion of the information is easier to perceive. This may enable theuser to more easily recognize the presented information despite theirfatigued state.

As used herein, many different ways to modify presentation ofinformation to become “easier to perceive” are contemplated byembodiments of the present invention. For example, though exampleembodiments described herein show an increase in font size of text orshapes, other modifications are envisioned (e.g., font style, color,contrast, texture, location on the display, etc.). Similarly, exampleembodiments may modify the presentation of the display in other ways tocause information to be easier to perceive (e.g., remove distractinginformation or cover-up environment information such as described withrespect to FIGS. 6A and 6B).

As noted above, in some embodiments, the apparatus 102, such as theprocessor 110, may be configured to cause a modification in at least oneof font size or color scheme of that at least a portion of theinformation being presented. For example, as illustrated in FIG. 5A, thecomputing device 200 has modified presentation of the information (shownin FIG. 4A) to become easier to perceive. In the depicted embodiment,the Mail icon 210′ has been enlarged and the title “Mail” 212′ has alsobeen enlarged with the font changed to all capital letters. Likewise,the Phone icon 225′ has been enlarged and the title “Phone” 227′ hasalso been enlarged with the font changed to all capital letters. In someembodiments, the degree of modification of certain information may beuniform across the display (e.g., the Mail icon 210′ and the Phone icon225′). Additionally, however, in some embodiments, the degree ofmodification may vary based on portions of the information beingpresented. For example, the title “Facebook” 213′ of the Facebook icon211′ may have not been enlarged as much as the title “Phone” 227′ of thePhone icon 225′. In such a regard, the apparatus 102 may be configuredto modify presentation of information on some portions of the displaydifferently from presentation of information on other portions of thedisplay. In some embodiments, the degree of modification may be based onan importance level of the information being presented or the operationand/or environment information corresponding to the information beingpresented. Such an importance level may be determined by the apparatus102 or may be pre-determined and/or pre-set, such as by the user.

FIG. 5B illustrates an example modification of presentation ofinformation for an apparatus (e.g., apparatus 102) associated with a car250 that includes a pass-through display 258. For example, the apparatusassociated with the car 250 has modified presentation of the information(shown in FIG. 4B) to become easier to perceive. In the depictedembodiment, the driving directions “STOP THEN TURN RIGHT” 275′ have beenenlarged and moved closer to steering wheel on the display 258.Additionally, the right turn arrow 277′ has also been enlarged on thedisplay 258. In such a manner, the user (e.g., the driver) may moreeasily perceive the information being presented.

In some embodiments, the apparatus 102, such as the processor 110, maybe configured to cause a modification by causing at least a portion ofthe information being presented on the display to be removed. In such aregard, the user may more easily perceive and/or focus on the remaininginformation being presented. For example, as illustrated in FIG. 6A, thecomputing device 200 has modified presentation of the information (shownin FIG. 4A) to remove the remaining icons (e.g., the Phone icon shown inFIG. 4A). These icons have been replaced by blank space 229 on thedisplay 208. Thus, the user may now more easily perceive the Mail icon210 and its corresponding title “Mail” 212. In this regard, theapparatus 102, such as the processor 110, may be configured to causeremoval of a portion of the information being presented in favor ofanother portion of the information being presented.

FIG. 6B illustrates an example modification of presentation ofinformation for an apparatus (e.g., apparatus 102) associated with a car250 that includes a pass-through display 258. In the depictedembodiment, the apparatus associated with the car 250 has modifiedpresentation of the information (shown in FIG. 4B) to cover-upenvironment information from the perspective of the user. The apparatushas blocked the light from the sun 293, which in some cases may removethe glare from the windshield 252. As illustrated in FIG. 6B, a portionof the sun 293 has been replaced by a blank space 229 on the display258. Thus, the driving directions “STOP THEN TURN RIGHT” 275 and theright turn arrow 277 may be easier to perceive.

Additionally or alternatively, in embodiments in which the apparatus 102comprises a pass-through display, the apparatus 102, such as theprocessor 110, may be configured to cause a modification of presentationof at least a portion of the information such that at least a portion ofthe environment information is easier to perceive. For example, FIGS.7A, 7B, and 7C illustrate example modifications or presentation of theinformation to enable the environment information to become easier toperceive. Embodiments of the present invention are not meant to belimited to these example modifications, as other modifications arecontemplated (e.g., highlighting the environment information, changingthe color of the environment information, etc.)

In some embodiments, the apparatus 102, such as the processor 110, maybe configured to modify presentation of information in the line of sightof the user to enable the user to more easily perceive the environmentinformation. For example, with reference to FIG. 7A, the apparatusassociated with the car 250 that includes a pass-through display 258 hasmodified presentation of information on the display 258 to enable a userto more easily recognize the stop sign 290 (shown in FIG. 4A). Inparticular, the apparatus associated with the car 250 has presented anenlarged sign 280 with the word “STOP” 282 located within, therebypresenting an over-sized stop sign to the user (e.g., the driver).

Similarly, with reference to FIG. 7B, the apparatus associated with thecar 250 that includes a pass-through display 258 has modifiedpresentation of information on the display 258 to enable a user to moreeasily recognize the need of the user to stop the car 250 (e.g., basedon the stop sign 290 shown in FIG. 4A). In particular, the apparatusassociated with the car 250 has presented the word “STOP” 283 over thestop sign 290 seen through the display 258.

In some embodiments, the apparatus 102, such as the processor 110, maybe configured to modify presentation of other information to enable theuser to more easily perceive the environment information. For example,with reference to FIG. 7C, the apparatus associated with the car 250that includes a pass-through display 258 has modified presentation ofinformation on the display 258 to enable a user to more easily recognizethe stop sign 290. In particular, both environment information (e.g.,the sun 293) and information being previously presented (e.g., thedriving directions “STOP THEN TURN RIGHT” 275 and the right turn arrow277) have been replaced and/or removed to enable the user to more easilyperceive the stop sign 290. As illustrated in FIG. 7C, a portion of thesun 293 has been replaced by a blank space 229 on the display 258 andthe driving directions 275 and right turn arrow 277 have been removed(e.g., near blank space 229′) from the display 258.

Embodiments of the present invention provide methods, apparatus andcomputer program products for modifying the presentation of informationbased on the attentiveness level of a user. Various examples of theoperations performed in accordance with embodiments of the presentinvention will now be provided with reference to FIGS. 8-9.

FIG. 8 illustrates a flowchart according to an example method formodifying the presentation of information based on the attentivenesslevel of a user according to an example embodiment 300. The operationsillustrated in and described with respect to FIG. 8 may, for example, beperformed by, with the assistance of, and/or under the control of one ormore of the processor 110, memory 112, communication interface 114, userinterface 116, or UI control circuitry 122. Operation 302 may comprisedetermining a first attentiveness level of a user. The processor 110,communication interface 114, user interface 116, sensor 118, and/or UIcontrol circuitry 122 may, for example, provide means for performingoperation 302. Operation 304 may comprise causing presentation ofinformation on a display. The processor 110, user interface 116, and/orUI control circuitry 122 may, for example, provide means for performingoperation 304. Operation 306 may comprise determining a secondattentiveness level of a user following determination of the firstattentiveness level of the user and while the information is beingpresented on the display. The processor 110, communication interface114, user interface 116, sensor 118, and/or UI control circuitry 122may, for example, provide means for performing operation 306. Operation308 may comprise causing, based on a relationship between the firstattentiveness level and the second attentiveness level, modification ofpresentation of at least a portion of the information on the display.The processor 110, user interface 116, and/or UI control circuitry 122may, for example, provide means for performing operation 308.

FIG. 9 illustrates a flowchart according to an example method formodifying the presentation of information based on the attentivenesslevel of a user according to an example embodiment 400. The operationsillustrated in and described with respect to FIG. 9 may, for example, beperformed by, with the assistance of, and/or under the control of one ormore of the processor 110, memory 112, communication interface 114, userinterface 116, or UI control circuitry 122. Operation 402 may comprisedetermining a first attentiveness level of a user. The processor 110,communication interface 114, user interface 116, sensor 118, and/or UIcontrol circuitry 122 may, for example, provide means for performingoperation 402. Operation 404 may comprise causing presentation ofinformation on a display. The processor 110, user interface 116, and/orUI control circuitry 122 may, for example, provide means for performingoperation 404. Operation 406 may comprise determining a secondattentiveness level of a user following determination of the firstattentiveness level of the user and while the information is beingpresented on the display. The processor 110, communication interface114, user interface 116, sensor 118, and/or UI control circuitry 122may, for example, provide means for performing operation 406. Operation408 may comprise causing modification of presentation of at least aportion of the information on the display in an instance in which thesecond attentiveness level is lower than the first attentiveness level.The processor 110, user interface 116, and/or UI control circuitry 122may, for example, provide means for performing operation 408.

FIGS. 8-9 each illustrate a flowchart of a system, method, and computerprogram product according to an example embodiment. It will beunderstood that each block of the flowcharts, and combinations of blocksin the flowcharts, may be implemented by various means, such as hardwareand/or a computer program product comprising one or morecomputer-readable mediums having computer readable program instructionsstored thereon. For example, one or more of the procedures describedherein may be embodied by computer program instructions of a computerprogram product. In this regard, the computer program product(s) whichembody the procedures described herein may be stored by one or morememory devices of a mobile terminal, server, or other computing device(for example, in the memory 112) and executed by a processor in thecomputing device (for example, by the processor 110). In someembodiments, the computer program instructions comprising the computerprogram product(s) which embody the procedures described above may bestored by memory devices of a plurality of computing devices. As will beappreciated, any such computer program product may be loaded onto acomputer or other programmable apparatus (for example, an apparatus 102)to produce a machine, such that the computer program product includingthe instructions which execute on the computer or other programmableapparatus creates means for implementing the functions specified in theflowchart block(s). Further, the computer program product may compriseone or more computer-readable memories on which the computer programinstructions may be stored such that the one or more computer-readablememories can direct a computer or other programmable apparatus tofunction in a particular manner, such that the computer program productcomprises an article of manufacture which implements the functionspecified in the flowchart block(s). The computer program instructionsof one or more computer program products may also be loaded onto acomputer or other programmable apparatus (for example, an apparatus 102)to cause a series of operations to be performed on the computer or otherprogrammable apparatus to produce a computer-implemented process suchthat the instructions which execute on the computer or otherprogrammable apparatus implement the functions specified in theflowchart block(s).

Accordingly, blocks of the flowcharts support combinations of means forperforming the specified functions. It will also be understood that oneor more blocks of the flowcharts, and combinations of blocks in theflowcharts, may be implemented by special purpose hardware-basedcomputer systems which perform the specified functions, or combinationsof special purpose hardware and computer program product(s).

The above described functions may be carried out in many ways. Forexample, any suitable means for carrying out each of the functionsdescribed above may be employed to carry out embodiments of theinvention. In one embodiment, a suitably configured processor (forexample, the processor 110) may provide all or a portion of theelements. In another embodiment, all or a portion of the elements may beconfigured by and operate under control of a computer program product.The computer program product for performing the methods of an exampleembodiment of the invention includes a computer-readable storage medium(for example, the memory 112), such as the non-volatile storage medium,and computer-readable program code portions, such as a series ofcomputer instructions, embodied in the computer-readable storage medium.

Many modifications and other embodiments of the inventions set forthherein will come to mind to one skilled in the art to which theseinventions pertain having the benefit of the teachings presented in theforegoing descriptions and the associated drawings. Therefore, it is tobe understood that the embodiments of the invention are not to belimited to the specific embodiments disclosed and that modifications andother embodiments are intended to be included within the scope of theinvention. Moreover, although the foregoing descriptions and theassociated drawings describe example embodiments in the context ofcertain example combinations of elements and/or functions, it should beappreciated that different combinations of elements and/or functions maybe provided by alternative embodiments without departing from the scopeof the invention. In this regard, for example, different combinations ofelements and/or functions than those explicitly described above are alsocontemplated within the scope of the invention. Although specific termsare employed herein, they are used in a generic and descriptive senseonly and not for purposes of limitation.

What is claimed is:
 1. A method comprising: determining, by a processor,a first attentiveness level of a user; causing presentation ofinformation on a display; determining a second attentiveness level ofthe user following determination of the first attentiveness level of theuser and while the information is being presented on the display; andcausing, based on a relationship between the first attentiveness leveland the second attentiveness level, modification of presentation of atleast a portion of the information on the display.
 2. The methodaccording to claim 1, wherein causing modification of presentation ofthe at least a portion of the information comprises causing modificationof presentation of the at least a portion of the information in aninstance in which the second attentiveness level is different than thefirst attentiveness level.
 3. The method according to claim 1, whereincausing modification of presentation of the at least a portion of theinformation comprises causing the at least a portion of the informationto be presented such that the at least a portion of the information iseasier to perceive.
 4. The method according to claim 1, wherein causingpresentation of the information on the display comprises causingpresentation of the information based on the first attentiveness levelof the user.
 5. The method according to claim 1, wherein determining thesecond attentiveness level comprises determining the secondattentiveness level based at least in part on output provided by atleast one of: a graze tracker, an accelerometer, or a heart ratemonitor.
 6. The method according to claim 1, wherein causingmodification of presentation of the at least a portion of theinformation comprises causing at least a portion of the informationpresented on the display to be removed.
 7. The method according to claim1, wherein causing modification of presentation of the at least aportion of the information comprises causing modification of at leastone of font size or color scheme of the at least a portion of theinformation.
 8. The method according to claim 1, wherein determining thesecond attentiveness level comprises determining the secondattentiveness level based at least in part on detecting at least one ofchanges in movement or behavior of the user's eyes or changes inmovement or behavior of the user's eyelids.
 9. An apparatus comprising aprocessor and a memory including computer program code, the memory andthe computer program code configured to, with the processor, cause theapparatus to: determine a first attentiveness level of a user; causepresentation of information on a display; determine a secondattentiveness level of the user following determination of the firstattentiveness level of the user and while the information is beingpresented on the display; and cause, based on a relationship between thefirst attentiveness level and the second attentiveness level,modification of presentation of at least a portion of the information onthe display.
 10. The apparatus of claim 9, wherein the memory and thecomputer program code are further configured to, with the processor,cause the apparatus to cause modification of presentation of the atleast a portion of the information by causing modification ofpresentation of the at least a portion of the information in an instancein which the second attentiveness level is different than the firstattentiveness level.
 11. The apparatus of claim 9, wherein the memoryand the computer program code are further configured to, with theprocessor, cause the apparatus to cause modification of presentation ofthe at least a portion of the information by causing the at least aportion of the information to be presented such that the at least aportion of the information is easier to perceive.
 12. The apparatus ofclaim 9, wherein the memory and the computer program code are furtherconfigured to, with the processor, cause the apparatus to causepresentation of the information on the display by causing presentationof the information based on the first attentiveness level of the user.13. The apparatus of claim 9, wherein the memory and the computerprogram code are further configured to, with the processor, cause theapparatus to determine the second attentiveness level by determining thesecond attentiveness level based at least in part on output provided byat least one of: a graze tracker, an accelerometer, or a heart ratemonitor.
 14. The apparatus of claim 9, wherein the memory and thecomputer program code are further configured to, with the processor,cause the apparatus to cause modification of presentation of the atleast a portion of the information by causing at least a portion of theinformation presented on the display to be removed.
 15. The apparatus ofclaim 9, wherein the memory and the computer program code are furtherconfigured to, with the processor, cause the apparatus to causemodification of presentation of the at least a portion of theinformation by causing modification of at least one of font size orcolor scheme of the at least a portion of the information.
 16. Theapparatus of claim 9, wherein the memory and the computer program codeare further configured to, with the processor, cause the apparatus todetermine the second attentiveness level by determining the secondattentiveness level based at least in part on detecting at least one ofchanges in movement or behavior of the user's eyes or changes inmovement or behavior of the user's eyelids.
 17. Computer program productcomprising a non-transitory computer readable medium having program codeportions means stored thereon, the program code portions being acomputer readable medium and configured when said program product is runon a computer or network device, to: determine a first attentivenesslevel of a user; cause presentation of information on a display;determine a second attentiveness level of the user followingdetermination of the first attentiveness level of the user and while theinformation is being presented on the display; and cause, based on arelationship between the first attentiveness level and the secondattentiveness level, modification of presentation of at least a portionof the information on the display.
 18. The computer program product ofclaim 17, wherein the program code portions are further configured whensaid program product is run on a computer or network device, to causemodification of presentation of the at least a portion of theinformation by causing modification of presentation of the at least aportion of the information in an instance in which the secondattentiveness level is different than the first attentiveness level. 19.The computer program product of claim 17, wherein the program codeportions are further configured when said program product is run on acomputer or network device, to cause modification of presentation of theat least a portion of the information by causing the at least a portionof the information to be presented such that the at least a portion ofthe information is easier to perceive.
 20. The computer program productof claim 17, wherein the program code portions are further configuredwhen said program product is run on a computer or network device, tocause presentation of the information on the display by causingpresentation of the information based on the first attentiveness levelof the user.